Translator for infants and toddlers

ABSTRACT

A system and method of interpreting communicative behaviors including actions, gestures and sounds produced by individuals, the system including a database of stored behavior codes, each behavior code having an associated explanation; means for accessing the database; means for recording a behavior performed by the individual; means for assigning an observed behavior code to the recorded behavior; means for comparing the observed behavior code to the stored behavior codes in the database of stored behavior codes; and means for retrieving the explanation associated with the stored behavior codes which match the observed behavior code. In the preferred embodiment the system further comprises means for recognizing events and correlating recognized events to the individual&#39;s behavior&#39;s.

FIELD OF THE INVENTION

[0001] This invention relates to a system and method of interpretinggestures and sounds produced by individuals in general, and, inparticular, to a system and method for recording, and processinggestures and sounds produced by individuals, comparing the processedrecordings to a database of interpreted gestures and sounds, andproviding caretakers of the individuals with interpretations of thegestures and sounds.

BACKGROUND OF THE INVENTION

[0002] The interpretation of nonverbal communication, i.e. gestures andsounds, is crucial for the care taking and treatment of nonverbalindividuals. Young children who have not yet learned to speak usebehaviors such as movements, gestures and sounds to express their needsand current state of emotions and well being. Nonverbal communication isoften the only means of expression for individuals with hearing loss,autism and other disorders and illnesses. Even for verbal individuals,nonverbal communication, if interpreted properly, can be used torecognize, diagnose and track treatment of medical disorders.

[0003] From time immemorial, parents, as an example, have been facedwith the challenge of determining the meaning behind gestures and soundsproduced by their nonverbal infants and toddlers. New, inexperiencedparents resort to trial and error to determine what need the child isexpressing. Experienced parents and care givers often encounternonverbal communicative behaviors with which they are not familiar.Child and parent are apt to experience frustration upon erroneousinterpretations of the child's nonverbal communications. The child'sfrustration masks the actual need being expressed. The parent'sfrustration is sensed by the child and will interfere with making aproper interpretation of the child's communications. The frustrationscan bring on undesirable behaviors on the part of the parent or child.Within time, the inexperienced parent will usually learn to intuitivelyrecognize patterns and correctly interpret their child's nonverbalcommunications with which they are familiar. In most cases thefrustration could have been avoided by providing the inexperiencedparent with the knowledge accumulated by experienced caretakers.However, there is no effective means for a parent to discover caretakerswho have experience with the particular behavior the parent is havingtrouble interpreting. Therefore, there is a need for a system and methodto provide caretakers of individuals with access to knowledge ofnonverbal behavior accumulated by experienced caretakers forinterpreting nonverbal communicative behavior of nonverbal individuals.

[0004] However, there are instances in which a nonverbal communicationis not contained within the knowledge already accumulated. The gesture,sound or movement recognized may be particular to the individual childor may indicate a condition which does not fall within the norm. It isprobable that the interpretation of the nonverbal communication, oncediscovered, will be of aide to a caretaker at some point in the future.Therefore, there is a need for a system and method for trackingpreviously unrecognized behaviors of individuals until an interpretationis discovered, and adding the newly interpreted behavior to the alreadyaccumulated knowledge.

[0005] There are instances in which an individual's nonverbalcommunication is related to current situational conditions such as anevent or an object proximate to the individual. It is difficult for acaretaker of the individual to account for the variable events orobjects which might be affecting the individual under their care, makingit difficult for the caretaker to associate the individual's behaviorwith an event or object. Therefore, there is a need for a system andmethod of associating events and proximate objects to the behaviors ofan individual.

[0006] Once a behavior occurs, it is difficult for care givers andcaretakers to compare it with other behavioral occurrences and toassociate it with situational conditions. Each behavior is a compositeof many factors. Tracking each factor of a behavior is confusing andtime consuming. A caretaker may not be capable of tracking proximateevents or objects for the purposes of association. with the behavioraloccurrences, particularly when the association is not obvious. Inaddition, the process of observing for the purpose of recognizingrepetitive behaviors takes care givers and caretakers away from theirother duties and responsibilities. Furthermore, it is difficult for aparent to be objective about their own child's behavior and to see itrealistically. Therefore, there is the need for a system and method forrecognizing a repetitive behavior and studying it through observing itrepetitively and analyzing it.

[0007] It may be impossible for a parent or care giver to observe orstudy a child's behavior from the same place where the child is located.Therefore, there is a need for a system and method to provide theability to observe the behavior of an individual from a location remotefrom the individual.

[0008] There are situations in which a caretaker cannot guard theindividual under their care from every danger in the environment. Forexample, the caretaker may be caring for more than one individual, maybe distracted while performing one of their duties, may be unfamiliarwith dangers in the environment, may be caring for an individual whotends to sneak and seek out trouble, etc. Therefore, there is a need fora system and method of alerting a caretaker when an individual undertheir care is exposed to potential danger.

[0009] In summary, there is a need for a system and method for providinga caretaker of an individual with the ability to identify nonverbalbehaviors of an individual under their care by comparing and matchingthe behavior with a collection of previously recorded nonverbalcommunicative behaviors, for associating nonverbal behaviors tosituational conditions, for recognizing patterns of behavior andassociating situational conditions to the behavioral patterns, and foralerting a caretaker when a nonverbal behavior poses a danger.

SUMMARY

[0010] It is an aspect of this invention is to provide a system and amethod for recognizing nonverbal behaviors performed by an individual,comparing the recognized behavior to a collection of known behaviors andlearning from the known behaviors what may be causing the recognizedbehavior and what can be done for the individual.

[0011] Another aspect of this invention is to provide a system and amethod for determining when a relationship exists between events andbehaviors.

[0012] Another aspect of this invention is to provide a system and amethod for detecting the occurrence of a dangerous event, and detectingbehaviors indicating that an individual is in serious trouble due to theoccurrence of a dangerous event and alerting a caretaker of theindividual appropriately.

[0013] Another aspect of this invention is to provide a system and amethod for providing caretakers of individuals with access toinformation collected relative to known behaviors and their causes, andfor providing the means for caretakers to communicate with one anotherand with experts in appropriate fields.

[0014] To accomplish the above and other aspects, a method is proposed,which, in the preferred embodiment, comprises the steps of observing andrecording a behavior of an individual; assigning a code to the observedbehavior; comparing the observed behavior code with a database ofbehavior codes associated with known behaviors; retrieving anexplanation associated with the known behavior codes which match theobserved behavior code; correlating related events to the observedbehavior; providing access to information stored about variousbehaviors, events and their relationships and providing means forcaretakers of individuals to communicate with one another and withappropriate experts. The method may further include the steps of sendingan alarm upon the occurrence of an event which poses a danger to theindividual; and sending an alarm upon recognition of behaviors of theindividual which indicate that the individual is in danger.

[0015] In addition, a system is proposed, which, in the preferredembodiment, comprises a recording means for recording a behaviorperformed by an individual, the recording means being in communicationwith a user interface; and an Analysis Provider in communication withthe user interface, the Analysis Provider having a database of behaviorcodes associated with known behaviors, and associated with each behaviorcode is an explanation and event data regarding events related with eachknown behavior. The Analysis Provider may further have a) a BehaviorCode Module for recognizing a recorded behavior and associating abehavior code with the recognized behavior, and b) a Behavior CodeMatching Module for finding behavior codes associated with knownbehaviors which match the recognized behavior code, and retrievinginformation associated with the known behavior code from the database.Furthermore, the Analysis Provider may include a Situational Analyzerfor identifying and tracking the occurrence of events which can becorrelated to the recognized behavior, for recognizing repeated behaviorpatterns of the individual behaviors, and for recognizing when an eventoccurs which poses a danger to the individual.

BRIEF DESCRIPTION OF THE FIGURES

[0016]FIG. 1 is a block diagram of the fundamental modules of apreferred embodiment of the present invention;

[0017]FIG. 2 is a diagram of a system according to the preferredembodiment of the present invention;

[0018]FIG. 3 is a flowchart of a method according to the preferredembodiment of the present invention;

[0019]FIG. 4 is a diagram of exemplary fields in a Behavior CodeDatabase record according to the preferred embodiment of the presentinvention;

[0020]FIG. 5 is an exemplary User Analysis Request Web page according tothe preferred embodiment of the present invention;

[0021]FIG. 6 is an exemplary Submit Recording Web page according to thepreferred embodiment of the present invention;

[0022]FIG. 7 is a block diagram of the Analysis Provider according tothe preferred embodiment of the present invention; and

[0023]FIG. 8 is a block diagram of the Situational Analyzer according tothe preferred embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

[0024] In the following description, the terms “server” and “database”are used in a generic functional sense. The term “server” should beunderstood within the client/server architectural model—the clientrequests a service, the server provides a service. The term “database”can be understood in its most broad definition as a data structurestored on one or more pieces of hardware. A “database” and “server” maybe depicted as one entity labeled a “database” which is inclusive of the“server”, yet it is to be understood that the “database” functions tostore data and the “server” functions to provide access to the“database”. Indeed, any of the servers or databases could be implementedusing a distributed network system, where the functional elements of aserver or database are not only distributed among nodes, but will oftenmigrate from node to node. On the opposite end of the spectrum, all ofthe servers and databases discussed below could be resident on onecomputer. However much of each server or database is implemented insoftware, firmware, or hardware being also open to many variations, asis well known in the art.

[0025] Furthermore, the terms “network” and “computer” are used in themost general sense. A “computer” is any computing means, from a singlemicroprocessor or micro controller to a computer system distributed overmultiple processing nodes. A “network” includes any means that connectscomputers. Thus, although the preferred embodiment uses the Internet,the nodes could connect through any other type of Intranet or networksuch as a central server with individual point-to-point connections, aLAN, etc.

[0026] The term “caretaker” refers to one who takes care of or givescare to another individual such as a parent, a baby sitter, a teacher, acare giver, a doctor, a therapist, a nurse, etc. The term “care giver”refers in particular to those involved in the temporary care of anindividual for providing a specialized service on a patient/clientbasis.

[0027] The terms “nonverbal behavior”, “nonverbal communication”, and“nonverbal communicative behavior” refer to movements, gestures andsounds (including gestures accompanied by words) produced by anindividual intentionally or unintentionally and which express a need,desire, or current state of emotional, mental or physical well being ofthe individual.

[0028] Other terms in the text are also to be understood in a genericfunctional sense, as would be known by one skilled in the art.

[0029] The procedure according to the preferred embodiment of thepresent invention is depicted in FIG. 1. In the first stage, BehaviorObservation 10, a nonverbal behavior or series of nonverbal behaviors isobserved. The second stage, Behavior Coding 20, involves associating acode with each behavior observed. The Behavior Code associated with eachbehavior observed is then compared to a collection of stored BehaviorCodes. Each Behavior Code in the collection of stored Behavior Codes hasan associated explanation regarding the cause of the behavior. When asuitable match is found, the explanation associated with the storedBehavior Code is retrieved and presented to the caretaker. If a suitablematch is not found the Behavior Code associated with the observedbehavior can be added to the collection of stored Behavior Codes. In thethird stage, Situational Analysis 30, events and objects are recognizedand correlated to the observed behavior.

[0030] Data gathered at the first stage is passed to the next stages.Although the second and third stages will be described in the preferredembodiment as being managed by one entity providing a service over theInternet, each stage could be handled by a separate entity. In otherwords, in the preferred embodiment, a single service provider ismanaging the second and third stages for subscribers. However, thesefunctions could be delegated to a service provider or company whoprovides the service of one or more of behavior observation, behaviorcoding or situational analysis. Furthermore, each or all of theseservices could be offered to a client by a single company or acombination of companies utilizing conventional methods other than theInternet for delivering information between the client and the company.

[0031]FIG. 2 is a diagram depicting the preferred embodiment of thesystem for providing the services shown in FIG. 1. As shown, acaretaker, who is a user 110 to the service provided by the system,operates recording means 112 for creating representations of nonverbalbehaviors performed by an individual 114. A user interface 116 isprovided for allowing the user to communicate through, for example,Internet 120 with a Nonverbal Behavior Analysis Provider Server (AP)118. The user operates the user interface 116 to submit selectedportions of the representations to the Analysis Provider 118 foranalysis.

[0032] The recording means 112 can comprise a device or manual means forrecording a representation of an individual's behavior. Therepresentation can be in any format such as textual, pictorial, graphic,digital or oral description or combination thereof. Devices employed asrecording means can be an analogue or digital camera, video camera,thermal imaging system, ultra sound imaging system or any system whichrecords a representation of the behavior of the individual. The devicecan be switched on and off manually or by automatic means such as theuser interface 116 or a motion detector. The device can store recordingsin files or submit them to the user interface 116 as they are beingrecorded.

[0033] The user interface 116 is any means for providing communicationbetween the user and the Analysis Provider 118. The communication can bein any format such as written, verbal or electronic. The communicationcan be man to man, man to machine or machine to machine. The userinterface 116 can further comprise carrier mail, a telephone, a wirelesstelephone, or a computer such as a personal or hand held computer. Inthe preferred embodiment, the user interface 116 and/or the recordingmeans 112 comprise computers and store the recordings in files. Thefiles can be reviewed and segments from the files can be selected fortransmission to the Analysis Provider 118. Furthermore, in the preferredembodiment, the user interface and/or the recording means are capable ofselectively allowing recordings to be transmitted to the AnalysisProvider 118 as they are being recorded. In the preferred embodiment theuser interface 116 communicates with the Analysis Provider 118 via theInternet 120. However, communication can be provided by an Intranetconnection or any known type of network. Alternatively, informationbeing transferred between computers comprising the user interface 116and the AP 118 is stored on a conventional storage device andtransported between the user interface 116 and the Analysis Provider118.

[0034] In the preferred embodiment, the user interface 116 and therecording means are in communication. This communication can be providedby cable connections or by wireless connections.

[0035] Alternatively, the behavior can be observed directly by theAnalysis Provider. This eliminates the need to form a representation ofthe behavior and provide it to the Analysis Provider.

[0036] The Analysis Provider can be a system comprised of personneland/or devices for receiving recordings, recognizing behaviors andevents, associating codes to the behaviors and events, and analyzing theevents and behaviors. In the preferred embodiment the Analysis Provider118 is a computer having or having access to a database of behaviorcodes.

[0037] The Analysis Provider 118 provides a service to a user 110. Theuser 110 can be a single individual or a plurality of individuals. Theuser 110 is generally a care giver or an assistant to a care giver ofthe individual whose behavior is being analyzed. The service provided bythe Analysis Provider 118 can be provided gratis or for a fee. The feecan be variable according to user requests, such as the extent ofanalysis and reporting requested by the user 110. The Analysis Provider118 can require that a user register with the service prior to using theservice.

[0038] In the preferred embodiment the service provided by the AnalysisProvider 118 includes receiving a representation of a behavior from auser, associating a code with the received behavior representation,comparing the received behavior code with a collection of storedbehavior codes, returning an explanation associated with the storedbehavior codes which match the received behavior code to the user.

[0039] A general outline of the basic steps performed by the AnalysisProvider 118 in the preferred embodiment of the inventive system isshown in FIG. 3. In step 201, the Analysis Provider receives arepresentation of the behavior to be analyzed from the user interface116. The representation is an image and sound recording comprising astring of frames. Alternatively, the representation can be only an imagerecording. The representation may be in analog or digital form. In step205 the recorded representation is formatted into a digital formatcomprising a string of digital frames and then compressed into a file.The file is compressed for preserving space, and for providing aschematic representation of the image which can be broken down foranalysis. In the preferred embodiment the file is compressed by samplingthe string of formatted digital frames at regular intervals and storingthe sampled frames with a time stamp. Each sampled frame is then storedas a multidimensional vector. Any known method can be employed fordigitizing and compressing the images. Sampling can be done at irregularintervals. Inactivity can be filtered out.

[0040] In step 210 a behavior code is associated with each behaviorrepresented in the compressed file. The Analysis Provider 118 recognizesgestures and uttered sounds which make up a behavior. Upon recognition,a code for each recognized gesture and sound is provided. When a gestureor sound is not recognized, a new code can be provided. The codes forthe gestures and sounds are used to form the behavior code. In anotherembodiment, only gestures are recognized and assigned codes for formingthe behavior code.

[0041] An example of a recognition system for identifying individualsaccording to their gestures and text independent speech is U.S. patentapplication Ser. No. 09/079,754 having the same assignee as the presentapplication, entitled “APPARATUS AND METHODS FOR USER RECOGNITIONEMPLOYING BEHAVIORAL PASSWORDS”, the disclosure of which is incorporatedherein by reference. In one embodiment of such a recognition system,behaviors comprised of a series of gestures are recognized. Each gestureis recognized by clustering similar frames of a compressed digital fileof images into a cluster, and grouping the clusters into strokes. Astroke is a series of frames which represent an individual performing aparticular gesture. The strokes are classified and labeled into gesturesymbols which are then interpreted to form a gesture unit.

[0042] The Analysis Provider 118 assigns a code to each gesture unit.The codes assigned to the series of gesture units which make up thebehavior are assigned to the behavior code for that behavior. Similarly,the sounds of the behavior are broken up into sound units which areassociated with a code and assigned to the behavior code. Thus, thebehavior code is comprised of at least one of a gesture component and asound component, each component being comprised of at least one unitcode.

[0043] The gesture units are further broken up into property units suchas which body part was moved, what type of movement was performed, theindividual's body position and the duration of the gesture unit. Eachproperty unit is assigned a code. The property codes assigned to theproperty units comprising a gesture unit are concatenated to form thegesture unit code. The sound unit will likewise be broken up intoproperty units such as sound pitch, sound volume, sound quality, and theduration of the sound unit, and may include units appropriate forautomating speech recognition (as known in the art). The property unitcodes assigned to the property units comprising a sound unit areconcatenated to form the sound unit code.

[0044] The behavior can be broken down into gesture units and soundunits in various ways, as for example according to predetermined timeintervals or according to the recognition of the end of a gesture orsound. The codes assigned to the gesture and sound units are integratedinto one behavior code comprising a string of integrated gesture unitand sound unit codes. Alternatively the gesture component and the soundcomponent of the behavior code can be maintained as separate componentsassociated with one behavior, each component comprised of a string ofcorresponding unit codes.

[0045] The behavior code can have a hierarchal structure such thatspecific gesture or sound unit codes are assigned priority levels. Thereare a variety of criteria according to which priority levels areassigned to behavior unit codes. Examples of criteria includechronological order, frequency of occurrence within the behavior codedatabase, degree of dominance, etc. . . . The degree of dominance refersto the degree to which a behavior unit is associated with the causes ofthe behavior. An expert, such as a behavioral psychologist, woulddetermine the degree of dominance of a behavior unit. For example, anexpert might attribute a higher degree to a behavioral unit representinghead banging than to a behavioral unit representing toe wiggling.

[0046] Similarly, the property unit codes are assigned selectablepriority levels. The priority levels are assigned according to thedegree to which a property unit distinguishes one behavior from anotherfor the purposes of finding a match when searching the behavior codesdatabase.

[0047] The priority levels assigned to behavior unit codes and propertyunit codes is selectable and can be determined by operators of thesystem and experts in the behavioral or medical fields. The selectionscan be made manually or automatically. The selections can be dynamic,such that the selections change as other factors change, such as newdata being added to the behavior code database. The priority is selectedin accordance with data stored in the individual and/or general behaviorcodes databases.

[0048] In step 215 a behavior code database containing a collection ofstored behavior codes is searched for behavior codes which match theassigned behavior code. The database is comprised of multiple records.Each record corresponds to one behavior code. Each record has multiplefields containing data pertinent to the record.

[0049] In the preferred embodiment the search will account for prioritylevels within the structure of the behavior codes. Behavior unit codeshaving a higher priority will be searched for prior to those of lowerpriority. Likewise, property unit codes having a higher priority will besearched for prior to those of lower priority.

[0050] In the preferred embodiment the criteria are specified for thedegree of similarity desired for determining a match between theassigned behavior code and behavior codes within the database. Thecriteria for match determination can be specified by operators of thesystem or by the user.

[0051] Upon determination of at least one match at step 218, step 220 isthen executed. At step 220 data is retrieved from each record having amatching behavior code. Included in the data retrieved from the matchingrecords is an explanation of the behavior represented by the matchingbehavior code. The explanation preferably is a textual segment enteredpreviously by a user of the system or by an expert consultant. At step230 a “match found” message comprising data retrieved from the matchingrecords is transmitted to the user.

[0052] In the instance where a match is not found, step 240 is executed.At step 240, with the user's consent, the assigned behavior code isadded to the database of behavior codes together with appropriate datato form a new record. The explanation field of the data record can beleft blank at this time and filled in at a later date if desired.

[0053] In the preferred embodiment, users of the service provided by theinventive system first subscribe to the service by registering with theAnalysis Provider 118. The user 110 provides the Analysis Provider 118with personal information which is entered into a subscriber database.An identification (subscriber ID) code is assigned and stored by thesystem for each user registered as a subscriber. Subscriber datarelating to financial information is kept confidential. The subscribercan have the option to allow or prevent access by other subscribers toinformation such as name, e-mail address, phone number and occupation.In other embodiments the service provided by the inventive system couldbe available to the public. Public users and subscribers would then havevarious levels of access to the information provided by the service.

[0054] The behavior code database can refer to a general behavior codedatabase for storing behavior codes associated with behaviors performedby a variety of individuals, and/or to an individual general behaviorcode database for storing behavior codes associated with behaviorsperformed by the individual. The individual database is preferablystored by the AP 118, or by the user interface 116. The means to searchand update the database is also preferably resident at either the AP 118or the user interface 116.

[0055] The behavior code database is structured in a variety of ways.FIG. 4 shows the structure of an exemplary data record of the generalbehavior code database. The behavior code record contains, for example,fields for an identification number (behavior ID) for the behaviorrepresented by the behavior code; links to other databases storing thefiles of the recordings of the behavior; a name for the behavior; thebehavior code; an identification number for the subscriber whooriginally entered the behavior into the database (originator ID); adate of original entry; a primary explanation for the behavior; a linkto a list of other explanations; a link to a list of event data relatedto the behavior, event data including, for example, the number ofoccurrences of each event associated with the behavior recorded, thedate and time of each event occurrence, a list of subscriber IDs whosubmitted a behavior related to the event; a link to a list ofrecommendations to the caretaker of the individual displaying thebehavior; a link to a list of similar behavior codes and their ID's.These fields are only examples, and the behavior code records maycontain many more or a subset thereof.

[0056] Preferably, the recordings of the behavior are stored and indexedaccording to the behavior ID. By storing the recordings, they can bereferred to for viewing or further analysis, and can be deleted when nolonger needed. The recordings are stored in any format such as theformat in which they were received or as a compressed file.

[0057] In the preferred embodiment, the lists are linked databaseshaving keys stored in fields of the records of the behavior codedatabase. Each list contains multiple records. For example, records inthe list for explanations, list for possible causes or list forrecommendations may contain multiple records each having fields for atextual entry, a subscriber ID for the subscriber who entered it, and adate of entry. The list of similar behaviors may contain multiplerecords each having a field for a behavior ID. A user viewing a recordcontaining a behavior ID or subscriber ID uses the ID to look up thebehavior or subscriber in their respective databases in order toretrieve more information.

[0058] Access to the Analysis Provider 118 is provided by the userinterface 116. In the preferred version a user 110 uses a web browserprogram, such as Netscape Navigator™, to provide access to the Internet120 for accessing the Analysis Provider 118. The user 110 exchangesinformation with the AP 118 via Web pages. The information beingtransmitted to and from the Analysis Provider 118 is protected forconfidentiality by employing encryption and decryption techniques suchas SSL (Secure Sockets Layer) if desired.

[0059] Prior to a user 110 submitting recordings of behaviors performedby an individual, the user 110 must provide the Analysis Provider 118with information about the individual 114. The information is stored bythe Analysis Provider 118 or by the user interface in an individualpersonal data file or database. The information may need to be updatedperiodically. The Analysis Provider 118 prompts or requires the user 110to update the information at specific time intervals, in particular forindividuals 114 whose condition is in flux, such as a growing child oran individual with an unstable medical history. Information which mustbe entered in the individual personal data file for registering anindividual 114 may include name, identification number assigned by thesystem, weight, height and dimensions of specific body parts. Themeasurements may be entered manually or entered automatically by ameasuring device. The AP 118 uses the physical measurements of theindividual 114 to scale the compressed file of the image for accurateassociation of a behavior code. Other information to be entered may berelated to allergies, disabilities, present medical condition, apicture, medical history, typical demeanor (appetite, mood, energylevel), current demeanor, etc. . . .

[0060] After the user enters or confirms updated information about theindividual, the user specifies criteria for the analysis of the imagesto be submitted. The AP then presents the user with questions to beanswered, which may depend on answers provided previously. The user isprompted to supply answers in various formats such as by filling in ablank field or by selecting from multiple choices. In the preferredembodiment, the user is presented with a User Analysis Request Web page,such as the page 500 shown in FIG. 5. Information fields that may be toolarge to show on the Web page are shown on separate pages or pop-upwindows by pressing the appropriate button. Lists are shown as scrollinglists or as pop-up windows.

[0061] At section 510 the user directs the analysis to extract from theimages submitted: a) a specific behavior, b) all behaviors or c)behaviors related to situational events. At section 515 the user choosesto further extract situational events or behaviors within a selectedchunk of time before and after each occurrence of a specified behavior.At section 520 the user describes the behavior being extracted. At field525 the user enters the behavior code for the specific behavior orbehaviors of the behavior pattern being extracted. The code is typicallyentered by keying the code in to field 525 or by selecting the code froma list of codes and descriptions 535. The description for any codeentered is displayed in field 530 in order for the user to confirm thatit is the code intended. At field 540 the user selects a series ofgesture and sound units separately which comprise a behavior. Thegesture unit is broken down, by example, into body part, movement, bodyorientation and duration of gesture. The user selects “Yes” or “No” ofinclusion of a sound unit. The sound unit, by example, is broken intosound description, intensity, pitch and duration of sound. The user thenindicates if a gesture or sound unit is simultaneous with the previousgesture or sound unit entered. The code and description which bestmatches the combination of gesture units selected will pop up in fields525 and 530 respectively. The user then enters a series of behaviorcodes which would constitute a behavior pattern. The user selects “Endof Behavior” to indicate when a series of behavior units has beenentered which constitute the behavior. The user then selects “EnterBehavior” to enter the entire series of behavior units.

[0062] The user specifies criteria for the degree of accuracy requiredin matching the behavior being extracted to behaviors stored in thedatabase. In this example, at field 550, the user selects a percentageof accuracy, however any method could be implemented. A minimum limit tothe degree of accuracy can be required. At field 560 the user specifiesthe maximum number of returned matches desired. The user can store hiscurrent selections as a default for future submissions at field 565, andsaves the current selections as a file at 570. The user then requeststhat the behaviors extracted be analyzed for determining if a patternexists at 575. The user next requests situational analysis of thebehaviors and events extracted at field 580 for determining if there isa relationship between the behaviors and events proximate to thebehavior or objects proximate to the individual. Finally, once the userhas entered the necessary information, the user submits the criteria foranalysis entered at 590.

[0063] Once the criteria for analysis has been submitted, the filecontaining the images selected to be analyzed must be submitted,although the order could be reversed.

[0064]FIG. 6 shows an exemplary screen 600 for a user to select theimages to be submitted to the AP for analysis. The user retrieves a fileof images stored by the recording means 112 or the user interface 116.The file to be retrieved is specified at field 605. The user specifies atime segment of the file to be retrieved at field 610. A view window 620is provided for viewing the images. As the user views the images infield 620 he marks the beginning and end of a segment by selecting beginand end segment marker buttons in section 625. In order to selectmultiple segments, the continue button is used to allow the user toselect successive segments. The erase marker button will allow a user toerase a begin segment or end segment marker, together with any markerpaired with it. The user controls the display in the view window 620using the controls in area 630. The user hits the “REC” (record) buttonto instruct the recording means to record and store its recordings inthe file specified at field 632.

[0065] The user enters events by selecting an event displayed in theview window 620, selecting “add event” and selecting the appropriateevent label from window 640. The time of occurrence will appear in thetime window 642. The user specifies the end of the event by selecting“end event”. Selecting a “danger” event will allow the user to enter anevent which poses a danger from window 644. The user is able to identifyobjects in the recording by using cursor 645 to point at or select theobject displayed in window 620, and selecting the appropriate objectlabel from window 650. Selecting a “danger” object allows the user toenter an object which poses a danger from window 655.

[0066] The user saves the selections made by selecting “save” andspecifying a file name at field 660. When the necessary information hasbeen entered, the user submits the selected segments to the AP foranalysis by selecting “submit selected segments” field 670.Alternatively, the user can choose to submit a live stream of images, asthey are being produced by selecting “submit live stream of images”field 675. The filename from which the images are retrieved must bespecified in field 680. When submitting a live stream of data, the userhas the option of selecting the option to “alert for danger” in field685. This will cause a screen to be presented to the user in which theuser must specify the nature of any dangers which might exist or occurin the vicinity of the individual. The user ends the stream of images byselecting “end stream” in field 690 with the filename specified in field680.

[0067] Now, the modules used by the AP 118 to analyze behaviorsrepresented in recordings submitted by the user 110, via the userinterface 116, will be described in detail with reference to FIG. 7. Theprocess begins when the request data and the recording data is sent tothe AP 118 over the Internet 120. As mentioned above, informationtransmitted via the Internet can be secured via encryption/decryptiontechniques which are commercially available. The incoming data,including request data entered by the user and recording data isreceived and handled by the Web Interface Server 705. All data beingsent to and from the AP 118 is handled by the Web Interface Server 705.

[0068] The Control Module 710 supervises the passage of control from onemodule to the next. The File Manager 715 is responsible for supplyingeach module with the proper file or segment of a file to be processed.The File Manager accesses databases of submitted recordings andcompressed, digitized formatted recordings. Recording databases areprovided for storing the recordings at the desired stages of processing.

[0069] The request data entered by the user 110 via web screens at theuser interface 116 is sent to the Control Module 710. The Control Modulesupervises the flow of control in order to fulfill the user's requests.Control is first sent to the Digitize and Sample Module 725. Therecording data file is sent by the file manager to the Digitize andSample Module 725.

[0070] The Digitize and Sample Module 725 converts any analogue data todigital data, and compresses the recording data by sampling the framesof the image data and sound data and saving the sampled frames in acompressed image file. Each sampled frame is time stamped. The Digitizeand Sample Module 725 can remove segments of image data in which thereis no activity, i.e. the frames are identical. Removal of inactivitysegments can be done either prior to, or post sampling.

[0071] Next, the compressed image file and control is sent to theScaling Module 730 in which data from the individuals's personal datafile, in the individual personal data database 731, pertaining to themeasurements of the individual, is accessed for providing a scalingfactor. The Scaling Module can either provide the scaling factor oractually scale the images. The scaled and compressed image file andcontrol is sent to the Noise Elimination Module 735 for eliminatingframes or portions of frames in which non-meaningful activity exists.The Noise Elimination Module 735 can consult a library of non-meaningfulactivities, or determine that an activity detected is non-meaningful.For example, devices in an infant's room or in a hospital room may be inmotion or produce noises which are not related to the behavior of theindividual. Examples of devices include a respirator, a clock ticking, arotating mobile in perpetual motion, etc. . . .

[0072] The control unit and file manager will pass control and send thescaled compressed recording file to a) the Behavior Coder (BC) Module740 when behaviors in the recording file are being analyzed, or to b)the Situational Analyzer 745 when events in the recording file are beinganalyzed.

[0073] When the scaled compressed recording file is received by theBehavior Coder Module 740, the compressed recording file is examined forrecognizing gestures and sounds which make up a behavior. The gesturescan be further grouped into gesture units, and the sounds furthergrouped into sound units. Each gesture and sound unit is assigned acode. The gesture unit code can comprise multiple parts such as bodypart moved, type of movement, position of body and duration of movement.The sound unit code can comprise multiple parts such as sounddescription, sound intensity and duration of sound. A Behavior Begin/EndDetector Module 750 is consulted by the Behavior Coder Module 740 fordetermining the end of one behavior and the beginning of the next. Thegesture and sound unit codes are combined to form a recognized behaviorcode for each behavior recognized. The combining of the gesture andsound units to form a code for a behavior can be done in a variety ofways. For example, a Synchronizer Module (not shown) can be used forsynchronizing the gestures and sound units to best model the actualbehavior. In a different method, the gesture units may be storedseparately from the sound units. Also, the gesture and sound units maybe stored consecutively according to chronological order, with anadditional gesture or sound unit code part for indicating when a unitcode is simultaneous with the previous unit code.

[0074] At this point, the Control Module 710 will pass control to theBehavior Code Matching Module 755 when each recognized behavior codeprovided by the BC Module 740 is to be compared to existing behaviorcodes for finding a match. On the other hand control will be passed tothe Behavior Series Matching Module 760 when a series of behaviors is tobe compared to existing series of behaviors for finding a match.

[0075] The Behavior Code Matching Module 755 determines if the userrequested that a specific behavior be searched for. In this case, therecognized behavior code is compared to the specific requested behaviorcode to determine if a match exists. The results are sent to the ReportGenerator 770. Otherwise, the Behavior Code Matching Module 755 searchesa Behavior Codes (BC) Database 765 to find a stored behavior code whichmatches the recognized behavior code.

[0076] When searching the Behavior Code Data Base 765 for a match, analgorithm is used for prioritizing the comparison of gesture or soundunits, or their parts. Many different algorithms could be used. Forexample, a first high priority class may be formed of all behavior codesin the Behavior Code Database having a first gesture unit with a bodypart moved which matches the body part moved of the first gesture unitin the recognized Behavior Code. A subclass can be formed from the firstclass by matching first gesture units having the same movement. Thesubclass can be further narrowed down by matching first gesture unitswith a body position and then narrowed further by matching gesture unitswith a duration within a specified threshold. The comparison can proceedby comparing gesture units in their chronological order. Subsequentsubclasses can be formed until matches have been found. In a differentalgorithm, certain types of body parts and movements would have priorityand are compared first, regardless of their chronological order.Duration of a movement or sound may or may not be compared until late inthe search or not at all.

[0077] Just as the recognized behavior codes can be determined by usinga module to synchronize the gesture and sound units or consider themseparately, the process of searching for a match can also involvecomparing gesture and sound units in a synchronized fashion orseparately form one another.

[0078] Determination of a match is made according to criteria which areselectable. The criteria is selected by the operators of the system, bythe system software, by the user or by any combination thereof. Forexample, the user may have specified that the criteria for a match be aperfect match, such that all of the components of all of the behaviorunits match. In another example, the user may have specified thatcriteria for a match be less than perfect, such as 80%. The degree towhich codes match one another can be calculated in various ways. Forexample, the match between a recognized behavior code and a storedbehavior code can be assigned a value. The match value can be assigned abeginning value of 100%, which is then decreased a specified amount foreach part of a behavior unit which doesn't match, or for matchinggesture and sound units which occur in a different order. The matchvalue can be decreased an even greater value when an entire behaviorunit in a stored code doesn't correspond to a behavior unit in theobserved code.

[0079] When control is transferred to the Behavior Series MatchingModule 760 it is determined if the user is searching for a specifiedbehavior series. If so, the recognized behavior codes as a string ofcodes are compared to the specified behavior series and the results aresent to the Report Generator 770 or to the SA Module 745 for furtheranalysis.

[0080] Preferably, the Behavior Code Database comprises a generalbehavior code database and an individual behavior code database. Thegeneral behavior code database stores codes associated with behaviors ofmultiple individuals. The individual behavior code database stores codesassociated with behaviors of the individual being observed. Theindividual behavior database is used, by example, for detecting patternsof the same behavior by the individual. The individual behavior databaseis stored at the user interface or at the AP, together with computerprogram code for executing searches and updating data related to theindividual database.

[0081] The Report Generator 770 receives results from the variousmodules and generates messages and reports which are sent to the uservia the Web Interface Server 705. The user specifies the type of reporthe wishes to receive, including how detailed it should be, uponregistering with the system or submitting a request. The report can beinteractive, including the option to proceed with the analysis. Forexample, the user may be prompted to change the criteria for a match viaa user Web screen and then repeat the search. This would be appropriatewhen no match was found or too many matches were found. The ReportGenerator 770 may query the user as to whether or not an unmatchedbehavior submitted by the user can be entered into the BC Database as anew behavior. The Report Generator 770 may further query the user as towhether or not information such as the User's subscriber ID and theindividual's ID can be entered into the general behavior code database.

[0082] Control is passed to the SA Module 745 for analysis of behaviorsand behavior patterns in relationship to events. The SA Module 745recognizes events and correlates events to behaviors or behaviorpatterns. An event refers to a) movement of an object, which is distinctfrom the individual; b) movement of an individual, relative to theobject; c) the presence of an object; d) an occurrence in the scheduleof the individual; e) an expected occurrence in the schedule of theindividual; or f) a detected change in the condition of the individual;in which the above cause a possible influence upon the behavior of theindividual. Individual event data such as the individual's schedule andcondition are entered into the system either manually by a user, orautomatically by a device such as a monitor. Individual event data suchas objects present in the images are entered into the system eithermanually by a user, or automatically by the system via recognitionmethods. An example of an automatic object recognition system foridentifying, monitoring, tracking and/or locating objects is U.S. patentapplication Ser. No. 09/239,845 having the same assignee as the presentinvention, entitled “A VIRTUAL MAP SYSTEM AND METHOD FOR TRACKINGOBJECTS”, the disclosure of which is incorporated herein by reference.

[0083] With reference to FIG. 8, when event data is entered for anindividual, it is stored in an Individual's Event (IE) Database 805which is linked to the Individual Personal Data Database for thatindividual. The IE Database 805 is comprised of records, each recordstoring information relative to an event. A General Event (GE) Library810 is provided for storing a variety of possible events which mightoccur. The GE Library 810 is comprised of records, each record storinginformation relative to an event.

[0084] In the inventive system, the SA Module 745 can be provided as anoptional unit. In the preferred embodiment, in which the SA Module 745is provided, a user 110 has the option to deactivate it. The SA Module745 can be activated by user request or by the system's internalinstructions.

[0085] When activated, control is passed to the SA module 745 by thecontrol module 710, along with specific instructions. The compressedimages to be processed are provided by the File Manager. The compressedrecordings are provided as a steady stream or as a finite segment.

[0086] The modules of the SA Module 745 will now be discussed in detail.An Event Identifier/Tracker (EI/T) Module 815 receives the compressedrecordings for detecting and identifying the occurrence of an event.Methods for detection of the occurrence of an event can include objectrecognition, tracking objects, tracking the individual for locationrelative to objects, tracking the time of an occurrence or expectedoccurrence in the individual's schedule and tracking the time of achange in the individual's condition. The Event Identifier/TrackerModule 815 compares the recognized object or event to records stored inthe Individual's Event Database 805 and the General Event Library 810.Events which were recognized, but could not be identified, aretransmitted to a Labeling Module 820 for labeling the event, creating arecord and entering it into the appropriate Event Database or Library.The event can be transmitted to the Labeling Module 820 in many formssuch as images, compressed images or codes. The Labeling Module 820 canbe an automatic intelligent system or a person for recognizing andidentifying the event.

[0087] Multiple occurrences of a behavior within a selected timeinterval surrounding an event may indicate a cause and effectrelationship. Likewise, the occurrence of an event within a selectedtime interval surrounding the performance of a behavior indicates acorrelation between the event and the behavior. An Event/BehaviorCorrelator Module (E/BC) Module 825 recognizes when a correlationbetween an event and a behavior exists and updates the related eventdata list field 335 of the behavior record stored in the BC Database 765for events correlated to the behavior. Event data stored in field 335includes the code associated with the correlated event, the number ofrecorded occurrences of each event and the time of the occurrencerelative to the behavior.

[0088] The SA 745 may be activated for the case in which a user hasrequested notification of occurrences of a specified event. Each eventrecognized is compared by an Event Matching Module 830 to the specificrequested event, in order to find a match. When a match is found,control will return to the Behavior Coder Module 740 for examining aspecified time interval preceding and/or following each occurrence of amatching event for recognizing behaviors within that time interval. TheBehavior Matching Module 755 will match the recognized behaviors tostored behaviors in the Behavior Code Database 765. The recognition andidentification of a behavior within the specified time intervalindicates that there may be a cause and effect relationship between thespecific event and the matched behavior code. Control returns to theEvent/Behavior Correlation (E/BC) Module 825 of the SA 745 for updatingthe related event data list field 335 of the matching behavior code. Forthe case in which the specific requested event was not recognized in theimages provided, control is returned to the Report Generator 770 forreturning a message to the user interface 116.

[0089] The SA 745 may further be activated for the case in which thesystem or user have requested identification of all recognized eventsoccurring within a segment of compressed images provided. The user mayhave requested that the system find a correlation between a specificbehavior or behavior pattern and any events which have occurred within aspecific time interval prior to or following the specified behavior. Asystem request is generated when the system recognizes that the relatedevent field 335, primary explanation field 320, or the relatedexplanation field 330 for a certain behavior code are still blank. Thesystem tries to generate an explanation for the behavior associated withthe behavior code by examining the associated compressed images of eachoccurrence of the behavior stored in the behavior code database. Aspecified time interval prior to and/or following each instance of thebehavior is searched for the occurrence of any events. The systemcorrelates any related events to the behavior.

[0090] The E/BC Module 825 updates the BC Database 765 by recording eachevent identified in the related events field 335 of the behavior codeassociated with the behavior being analyzed. Alternatively, the E/BCModule 825 uses intelligent processing to enter only the events whichare likely to be related. The E/BC Module 825 looks for a pattern ofevents and makes a determination as to which events are most likelyrelated to the behavior. The results are then sent to the ReportGenerator 770.

[0091] A Behavior Pattern Recognizing (BPR) Module 830 examines theIndividual BC Database for patterns of behavior for the purpose ofgrouping behaviors together which typically occur approximate oneanother. Upon determining that a group of behaviors are related, the BPRModule 830 examines the related event field 335 of each of the behaviorswithin the group. The BPR Module 830 determines if there are commonrelated events amongst the group indicating that an event may be thecause of a number of related behaviors. The results are then sent to theReport Generator 770.

[0092] The SA 745 further functions to recognize when an event occurswhich presents potential danger to the individual. Examples of dangerousevents include exposure to a severe allergen, ingestion of a chokinghazard, the individual makes contact with a sharp object or a poisonoussubstance, the individual approaches an electric or fire hazard, aserious medical condition is sensed by a monitor, etc. . . . The eventrecords in the Individual Event Database and the General Event Libraryinclude a field for indicating which events pose a danger. Upondetection of a dangerous event, the results are sent to the ReportGenerator for transmitting an alert to the user or caretaker of theindividual. Furthermore, control is passed to the Behavior Coder Modulefor determining if a behavior has occurred which indicates that theindividual is in trouble. Trouble behaviors being searched for couldinclude typical choking gestures or sounds, inactivity, seizure,screaming, severe crying, etc. . . .

[0093] Subscribers can access the subscriber, behavior code and imagedata bases to look up information. Subscribers may be allowed to addinformation to behavior records such as adding to the lists for possiblecauses, explanations and recommendations. Various levels of access toinformation can be assigned to subscribers and public users.

[0094] In the preferred embodiment the system provides the userinterface with the software necessary for allowing the user to exchangeinformation with the AP via the Web *Screens. Via the Web screens theuser can browse and search the general databases for specificinformation. The User Query Module 720 handles the user requests,accesses the appropriate database, retrieves the information requestedand provides the information to the Web Interface Server for returningthe requested information to the user interface. In addition the usersof the system can communicate with other users and experts. Furthermore,users and experts can communicate with one another interactively and ingroup formats such as chat rooms.

[0095] While the invention has been described with respect to thepreferred embodiment, it should be understood that the invention is notlimited to this embodiment, but, on the contrary, the invention isintended to cover all modifications, equivalents, and alternativesfalling within the spirit and scope of the invention as defined by theappended claims.

In the claims:
 1. A method for interpreting nonverbal communicativebehavior of an individual, the method comprising the steps of: (a)observing and recording a behavior performed by the individual; (b)acquiring access to a collection of behavior codes, each behavior codehaving an associated explanation; (c) assigning an observed behaviorcode to the observed behavior; (d) comparing the observed behavior codeto the collection of behavior codes; (e) retrieving the explanationassociated with each behavior code of the collection which matches theobserved behavior code.
 2. The method according to claim 1, furthercomprising the step of storing the observed behavior.
 3. The methodaccording to claim 2, wherein the recorded behavior is stored insegments.
 4. The method according to claim 3, wherein a selected segmentis transmitted by a user via a user interface to a computer, wherein thecomputer assigns the observed behavior code, searches the collection ofbehavior codes for matches by comparing the observed behavior code tothe collection of behavior codes, and retrieves the explanationassociated with each behavior code which matches the observed behaviorcode.
 5. The method according to claim 2, wherein the recording iscomprised of a string of frames, and the selected segment received bythe computer is compressed into a compressed file by sampling the framesof the selected segment, and digitizing and formatting the sampledframes.
 6. The method according to claim 5, wherein the recording isfurther comprised of a string of audio frames and image frames.
 7. Themethod according to claim 5, wherein recorded inactivity is filtered outand not included in the digital compressed file.
 8. The method accordingto claim 1, wherein the step of assigning an observed behavior code tothe observed behavior further comprises: recognizing behavior unitswhich comprise the behavior; assigning a behavior unit code to eachbehavior unit; combining the behavior unit codes to form an observedbehavior code.
 9. The method according to claim 8, wherein the behaviorunits comprise gesture units and sound units; and wherein the behaviorunit codes comprise gesture unit codes and sound unit codes.
 10. Themethod according to claim 9, wherein the gesture unit codes and soundunit codes are synchronized to form the observed behavior code.
 11. Themethod according to claim 9, wherein the behavior unit codes areconcatenated in chronological order, and each behavior unit codeindicating when the associated behavior unit occurred simultaneouslywith the behavior unit associated with the preceding behavior unit code.12. The method according to claim 1, wherein when a match is not foundfor the observed behavior code, the observed behavior code is added tothe collection of stored behavior codes as a new code.
 13. The methodaccording to claim 12, wherein the collection of stored behavior codesis a database.
 14. The method according to claim 13, wherein an ID isassociated with a user and an ID is associated with the individual, andthe user has the option of allowing the user ID and the individual ID tobe entered into the database in association with a new code, when a newcode is added, and in association with any matching stored behavior codein the behavior code database.
 15. The method according to claim 8,wherein the behavior units comprising the observed behavior areprioritized according to one of: chronological order, frequency ofoccurrence in database and degree of dominance.
 16. The method accordingto claim 15, wherein the step of comparing the observed behavior code tothe collection of behavior codes further comprises: searching thecollection of behavior codes for behavior codes having behavior unitsmatching the behavior units of the observed behavior code, in order ofthe priority of the behavior unit codes in the observed behavior code.17. The method according to claim 16, wherein the step of comparing theobserved behavior code to the collection behavior codes furthercomprises: synchronizing the comparison of the gesture units and thesound units of the observed behavior to the gesture units and the soundunits of the stored behaviors.
 18. A method for interpreting nonverbalcommunicative behavior of an individual, the method comprising the stepsof: observing a behavior pattern comprised of multiple behaviorsperformed by the individual; acquiring access to a collection ofbehavior pattern codes, each behavior pattern code comprised of a stringof behavior codes; assigning a behavior pattern code to the behaviorpattern, formed by stringing together the codes associated with themultiple behaviors; comparing the observed behavior pattern code to thecollection of behavior pattern codes; reporting the results of thecomparison.
 19. The method according to claim 18, wherein the collectionof behavior codes and the collection of behavior pattern codes areacquired according to past behaviors and behavior patterns performed bythe individual.
 20. The method according to claim 18, wherein thecollection of behavior codes and the collection of behavior patterncodes are acquired according to past behaviors and behavior patternsperformed by multiple individuals.
 21. The method according to claim 18,wherein the method further comprises the steps of: recognizing events;assigning an event code to each recognized event; and correlatingrecognized events to one of behaviors and behavior patterns.
 22. Themethod according to claim 21, wherein an event causes an influence uponthe behavior of the individual and comprises at least one of: a)presence of an object, b) movement of an object, c) movement of theindividual relative to an object, d) an occurrence in the schedule ofthe individual, e) an occurrence expected to take place in the scheduleof the individual, and f) a detected change in the condition of theindividual.
 23. The method according to claim 21, wherein the step ofcorrelating recognized events comprises: associating event data witheach behavior code, wherein the event data comprises the event code ofeach event which has been correlated with the behavior associated withthe behavior code, the time of the occurrence of the event, the numberof occurrences of the event; and updating the event data associated withthe behavior code in the collection of behavior codes which matches anobserved behavior code upon the recognition of an event occurring withina selected time interval preceding and following the performance of thebehavior.
 24. The method according to claim 23, wherein the step ofcorrelating recognized events further comprises: analyzing event datafor a selected behavior code from the collection of behavior codes, todetermine which event occurrences are related to a behaviors representedby the selected behavior code according to the number of occurrences andthe time interval between the event and the occurrence of the behavior.25. The method according to claim 23, wherein the step of correlatingrecognized events further comprises: determining which observedbehaviors performed by an individual occur within a selected timeinterval of one another repetitively, for establishing a behaviorpattern; comparing correlated events for each of the observed behaviorsincluded in the behavior pattern, and determining which of thecorrelated events are the same.
 26. The method according to claim 18,wherein the method further comprises the steps of: recognizing theoccurrence of an event which could pose a danger to the individual;transmitting an alert upon the recognition of a danger posing event. 27.The method according to claim 26, further comprising the steps of:associating a danger alert to behavior codes in the collection ofbehavior codes for behavior codes associated with behaviors whichindicate that an individual is in danger; upon comparing observedbehavior codes associated with behaviors observed following a dangerposing event occurred, retrieving existing danger alerts for eachmatched behavior code; and transmitting an alert upon retrieval of adanger alert.
 28. A system for interpreting nonverbal communicativebehavior of an individual, the system comprising: a database of storedbehavior codes, each behavior code having an associated explanation;means for accessing the database; means for recording a behaviorperformed by the individual; means for assigning an observed behaviorcode to the recorded behavior; means for comparing the observed behaviorcode to the stored behavior codes in the database of stored behaviorcodes; and means for retrieving the explanation associated with thestored behavior codes which match the observed behavior code.
 29. Asystem for interpreting nonverbal communicative behavior of anindividual, the system comprising: means for recording a behaviorperformed by the individual; a user interface in communication with therecording means for receiving recordings; and an Analysis ProviderServer in communication with the user interface, said Analysis ProviderServer receiving a recording of a behavior; said Analysis ProviderServer comprising; means for assigning a recorded behavior code to therecorded behavior; a Behavior Code Database for storing known behaviorcodes associated with known behaviors, each known behavior code havingan associated explanation associated thereto; and means for comparingthe recorded behavior code with the known behavior codes in the BehaviorCode Database and retrieving the explanation associated with each of theknown behavior codes which match the recorded behavior code.
 30. Thesystem according to claim 29, wherein the received recording is a fileof a series of frames; and wherein the means for assigning a recordedbehavior code to the recorded behavior comprises: a Digitize and SampleModule for sampling the frames of the received recording at regularintervals and forming the sampled frames into a digital compressedformatted File of Sampled Frames with each sampled frame having a timestamp indicative of the relative time it was recorded.
 31. The systemaccording to claim 30, wherein the Analysis Provider Server furthercomprises an Individual Personal Data database for storing data aboutthe individual, the data comprising identification, age, bodymeasurements and voice characteristics.
 32. The system according toclaim 31, wherein the Digitize and Sample Module is in communicationwith a Scaling Module for applying a scaling factor selected accordingto the data stored in the Individual Personal Data database to thesampled frames of the digital compressed file for providing astandardized version of the sampled frames to the means for assigning arecorded behavior code to the recorded behavior.
 33. The systemaccording to claim 30, wherein the Digitize and Sample Module is incommunication with a Noise Elimination Module for eliminating from thesampled frames recorded activity which is not meaningful to behaviors ofthe individual.
 34. The system according to claim 30, wherein the meansfor assigning a code to the recorded behavior comprises: a BehaviorCoder (BC) Module for recognizing behavior units which comprise thebehavior, assigning a behavior unit code to each behavior unit, andcombining the behavior unit codes to form a recorded behavior code. andwherein the known behavior codes are comprised of behavior units codes.35. The system according to claim 34, wherein the behavior unitscomprise gesture units and sound units, and wherein the behavior unitcodes comprise gesture unit codes and sound unit codes.
 36. The systemaccording to claim 35, wherein the BC Module is in communication with aSynchronizer Module for synchronizing the gesture unit codes and thesound unit codes to form the recorded behavior code.
 37. The systemaccording to claim 35, wherein the BC Module is in communication with aBehavior Begin/End Detector Module for determining the end of therecorded behavior and the beginning of a subsequent recorded behavior.38. The system according to claim 35, wherein the BC Module concatenatesthe behavior unit codes in chronological order, with each behavior unitcode indicating when the associated behavior unit occurredsimultaneously with the behavior unit associated with the previousbehavior unit code.
 39. The system according to claim 35, wherein thebehavior unit codes are comprised of property unit codes, wherein theproperty unit codes of a gesture unit code comprise a body part unitcode, a movement type unit code, a body position unit code and aduration unit code.
 40. The system according to claim 39, wherein theproperty unit codes of a sound gesture code comprise a sound descriptionunit code, a sound intensity unit code, a sound pitch unit code and aduration period code.
 41. The system according to claim 39, wherein thebehavior unit codes comprising the recorded behavior code areprioritized according to one of: chronological order, frequency ofoccurrence in database and degree of dominance.
 42. The system accordingto claim 41, wherein the means for comparing the recorded behavior codewith the known behavior codes stored in the database comprises: aBehavior Code Matching (BCM) Module for searching the Behavior CodeDatabase for behavior codes having behavior units having property unitsmatching the property units of the observed behavior code.
 43. Thesystem according to claim 42, wherein the Behavior Code Database issearched according to the priority of the behavior unit codes and theproperty unit codes.
 44. The system according to claim 41, wherein themeans for comparing the recorded behavior code with the known behaviorcodes stored in the database synchronizes the comparison of the gestureunit codes and the sound unit codes of the recorded behavior code to thegesture unit codes and the sound unit codes of the stored behaviorcodes.
 45. The system according to claim 42, wherein a user enters, viathe user interface, a requested series of behavior codes; and theAnalysis Provider Server comprises a Behavior Series Matching (BCM)Module for determining when the individual performs a series ofbehaviors which correspond to the requested series of behavior codes.46. The system according to claim 45, wherein the Analysis ProviderServer further comprises a Web Server for receiving data from andtransmitting data to the user interface.
 47. The system according toclaim 46, wherein the Analysis Provider Server further comprises aSubscriber Database for storing data relative to the users of the systemcomprising an identification number, personal data and financial data.48. The system according to claim 47, wherein the Behavior Code Databasecomprises a record for each behavior code, each record comprising afield for storing: the behavior code; an Identification Number for thebehavior code; a link to the file of the received recording; a link tothe File of the Sampled Frames; a primary explanation for the behaviorcode; a link to a list of similar behavior codes; a link to a list ofrelated explanations for the behavior code with dates of entry andidentification numbers for the submitting users; recommendations relatedto the behavior code with dates of entry and identification numbers forthe submitting users; an identification number for the user who firstsubmitted the behavior code; an identification number for theindividual; and an indicator that the behavior code indicates that theindividual is in danger.
 49. The system according to claim 48, whereinthe Analysis Provider Server further comprises an Individual BehaviorCodes Database for storing records for at least one of behavior codes orseries of behavior codes performed by the individual, wherein eachrecord comprises a field for storing: the behavior code; a key numberfor the behavior code; a link to the file of the received recording; alink to the File of the Sampled Frames; a primary explanation for thebehavior code; a link to a list of similar behavior codes; a link to alist of related explanations for the behavior code; recommendationsrelated to the behavior code; and an indicator that the behavior codeindicates that the individual is in danger.
 50. The system according toclaim 49, wherein the Analysis Provider Server further comprises anIndividual Event Database for storing events relating to the schedule,environment and condition of the individual, each event having an eventcode and a danger indicator which indicates if the event poses a dangerto the individual.
 51. The system according to claim 50, wherein theAnalysis Provider Server further comprises a Situational Analyzer (SA)Module for correlating events to recorded and stored behavior codes andseries of behavior codes; wherein the events influence the behavior ofthe individual and comprise at least one of: a) a presence of an object,b) movement of an object, c) movement of the individual relative to anobject d) an occurrence in the schedule of the individual, e) anoccurrence expected to take place in the schedule of the individual, orf) a detected change in the condition of the individual.
 52. The systemaccording to claim 51, wherein the Situational Analyzer Modulecomprises: a General Event Database; an Event Identifier/Tracker (EI/T)Module for tracking objects and recognizing the occurrence of events inthe File of Sampled Frames; and an Event/Behavior Correlator (E/BC)Module for correlating recognized events with behaviors recorded in theFile of Sampled Frames.
 53. The system according to claim 52, whereinthe EI/T Module recognizes objects in the File of Sampled Frames andidentifies the objects by looking them up in the Individual EventDatabase and the General Event Database.
 54. The system according toclaim 52, wherein the SA Module further comprises a Labeling Module foridentifying events; and wherein the EI/T Module queries the LabelingModule when a recognized event is not identified, and upon receiving asuccessful identification of the event from the Labeling Module, theEI/T adds the event into at least one of the Individual Event Databaseor General Event Database.
 55. The system according to claim 49 whereineach record at least one of the Behavior Code Database and theIndividual Behavior Code Database further comprises a field for storinga link to event data, wherein the event data comprises an event code foreach event which has been correlated with the behavior code, the time ofthe occurrence of each correlated event relative to the occurrence ofthe behavior associated with the behavior code, and the number ofoccurrences of each correlated event.
 56. The system according to claim52, wherein the E/BC Module correlates recognized events to the behaviorrecorded in the File of Sampled Frames by determining when a recognizedevent occurs within a selected time interval preceding and following theperformance of the behavior; and updates the associated event data ofthe behavior code associated with the performed behavior in at least oneof the Behavior Code Database and the Individual Behavior Code Database.57. The system according to claim 52, wherein the Individual EventDatabase and the General Event Database include a field for each eventwhich indicates when the occurrence of the event poses a threat to theindividual; and wherein the SA Module causes an alert to be transmittedto a caretaker of the individual upon recognition of the occurrence ofdanger posing event.
 58. The system according to claim 51, wherein theSA Module further comprises a Behavior Pattern Recognizing (BPR) Module,wherein said BPR Module recognizes the repetitive occurrences of seriesof behavior codes for establishing behavior patterns and recognizes theoccurrence of a behavior pattern performed by the individual.
 59. Thesystem according to claim 29, wherein the AP Server provides the userinterface with interactive web pages to enable a user to submit one of alive stream of recordings and a selected recorded segments; and toenable the user to request that the AP Server recognize at least one ofa specific behavior, a specific series of behaviors, a specific event,all behaviors and all repeated behavior pattern.
 60. The systemaccording to claim 59, wherein the AP server enables the user to requestthat the AP Server correlate at least one of a behavior to a specifiedevent, an event to a specified behavior, an event to a specified seriesof behaviors, and correlation of events to all recognized behaviors. 61.The system according to claim 59, wherein the user specifies a behaviorand a series of behaviors by entering, at a Web Page provided, one ofthe behavior codes for each behavior, the identification number for eachbehavior and behavior units comprising the behavior.
 62. The systemaccording to claim 50, wherein the user enters events into theIndividual Event Database, at a Web Page provided, by one of: enteringdescriptions of each event; viewing the recording in a view window on aWeb Page, selecting an object displayed in the view window andidentifying the object; and viewing the recording in a view window on aWeb Page and entering the event which occurred at the time shown in theview window.
 63. The system according to claim 29, wherein the AnalysisProvider Server further comprises a Query Module for processing userrequests to enter and access data stored in the AP Server.
 64. Thesystem according to claim 63, wherein the Analysis Provider Serverfurther comprises a Report Generator Module for transmitting messages,results from submitted requests and alerts to the user interface.
 65. Asystem for interpreting nonverbal communicative behaviors of anindividual comprising: a camera for recording behaviors of anindividual; a first computer in communication with the camera forselecting a segment of the recorded behavior; a second computer incommunication with the first computer for prestoring a collection ofcoded definitions of behaviors, each coded definition having anidentification code and an explanation; receiving the selected segmentfrom the first computer; converting said segment into a digital format;sampling said formatted segment at regular intervals; storing saidsamples; converting said samples of said formatted segment into a codeddefinition of said behavior; comparing said coded definition of saidreceived segment of the recorded behavior to the pre-stored collectionof coded definitions of behaviors; returning to said first computer theidentification code and explanation of each of the pre-stored codeddefinitions which match the coded definition of the received segment;and if no matches were found; adding the definition code of the receivedsegment to the collection of pre-stored coded definitions; providing theadded coded definition with an identification code and a blankexplanation code; and returning the identification code of the addedcoded definition to the first computer.
 66. An article of manufacturecomprising: a computer usable medium having computer readable programcode means embodied therein for interpreting a recorded nonverbalcommunicative behaviors of an individual, the computer readable programcode means in said article of manufacture comprising: computer readableprogram code means for assigning a label to the recorded behavior;computer readable program code means for accessing a database ofbehavior labels, each behavior label having an associated explanation;comparing the label for the recorded behavior to the labels in thedatabase of behavior labels; retrieving the explanation associated witheach behavior label of the database which matches the recorded label;updating the database by adding the recorded label to the database if amatch was not found; computer readable program code means forcorrelating events associated with and included in the recorded behaviorto the recorded behavior; and computer readable program code means forproviding Graphic User Interface (GUI) for a user to submit recordingsto the computer usable medium and request and receive information fromthe usable medium.
 67. The method according to claim 2, furthercomprising the step of: submitting the stored observed behavior to aservice provider, and wherein the service provider performs steps(b)-(e) for a fee.